To overcome the shortcomings of Cross-Media Relevance Model (CMRM) whose efficiency and effectiveness are low, an improved CMRM was proposed. Based on the improved smoothing method for textual words, the improved CMRM simplified the feature representation and similarity computation which made the measure of relationship between image and image more accurate. The experimental results on the Corel5k dataset show that the proposed approach can significantly improve annotation efficiency. The performance of the improved CMRM is almost three times as good (in terms of mean F1-measure) as original CMRM, also, better than some previously published high quality algorithms such as famous Multiple Bernoulli Relevance Model (MBRM) and Supervised Multiclass Labeling (SML).